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Abstract 


This paper reports on research conducted on the effectiveness and relevance 
of authentic assessment in a Mobile Application Development course, 
answering the research question “How practical is authentic assessment in 
Digital Media courses?” 


To facilitate this research, students were provided a set of tasks that consisted 
of developing a Mobile application for the Bundaberg Health and Wellbeing 
program. They were required to develop a working prototype mobile 
application that measures health related routine using the latest programming 
tool and APIs as used in industry. The mobile application is then used in the 
Bundaberg Health and Wellbeing program as a tool to monitor the progress 
of group members, providing a more authentic experience that connects and 
relates to the real needs and expectation of a client. 


A survey was then conducted to assess how students felt about the 
assessment piece and how it compared to other assessment they had done 
before. Overall, this indicated that most of the students favoured the 
introduction of authentic assessment in the course program; only a portion of 
the students feels otherwise. Based on this, the paper will conclude with 
strategies to encourage future technology-based courses in digital media 
courses to be based on authentic assessment. 


Introduction 


Assessment has always been a part of the educational landscape, serving many different purposes and 
coming in many various forms. In digital media courses, authentic assessments are essential for 
students to develop a 21st-century skillset, inclusive of the practical skills needed for their career. 
Digital media students need to respond to real-world tasks, and should be able to create industry 
standard digital products with a purpose and for an audience (Pearson & Valencia, 1987). 


In recent years, the development of Digital technologies has increased tremendously. It has altered the 
way people communicate. The growing use of digital technologies means the usage of digital 
information over the internet has grown exponentially. It comes in various forms such as text, 
graphics, animation, video or audio. There were many platforms that were being used to deliver the 
data; one of the traditional ways of transmitting the data is using the desktop computer. It has changed 
over the years, Mobile phone or smartphone has become the latest platform that is being used to 
transmit and download the data. 


The purpose of this paper is to report on research conducted through the use of authentic assessment as 
part of the course materials in a Mobile Application Development course, a Digital Media program at 
an Australian university. The objectives of this research are to study the effectiveness and relevance of 
the authentic assessment introduced in this course, to answer the research question “How practical is 
an authentic assessment in Digital Media courses?” A pilot study was conducted to a batch of student 
who did an authentic assessment in this course. The students were asked to develop a working Mobile 
application for the Health and Well-being program in Bundaberg Australia. After it had been done, a 
survey was conducted to see how the students respond towards the assessment. Based on this analysis, 
the paper provides recommendations for improving authentic assessment integration in classrooms. 
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Context 


The Bachelor of Digital Media in the School of Education and the Arts program at Central Queensland 
University is an undergraduate program which offers various range of expertise in Digital media 
landscape. The program itself is unique, and it offers the students multiple courses that develop 
comprehensive skill in digital media and journalism. The Bachelor of Digital Media program contains 
courses like Animation, Video development, Interactive games development, 2D and 3D graphics 
development and Mobile Application Development. 


The paper will focus on the assessment on one of its unit, the unit in the program that was used as a 
pilot study was Mobile Application Development. The Mobile application development course is a 12 
weeks course program emphasizing in the development of a Mobile applications prototype. The 
Mobile Application Development course teaches a student on how to develop a mobile application for 
mobile devices such as Mobile phone and tablets. The combination of theory and practical approach in 
this course makes it almost near to the actual industry expectation of the end product. The students 
were expected to know the latest market trend for the Mobile application and the technologies that 
come with it. 


The students that enrolled in this course are mainly third-year students who have a solid background 
on programming and digital media. Other students from other programs are normally interested in this 
particular course, and they would enroll it as an elective subject. The students are expected to be 
competent and knowledgeable in using standard programming language. In addition to that, the 
students are expected to have knowledge of external programing language to such as APIs to enhance 
the usability of their Mobile application. 


Since the introduction of the Mobile application subject, this course has generated more than 70% 
increase in its enrolment. The expectation in this course is rather high as it opens a pathway for a 
student to develop an actual Mobile application once they have completed this course. The technology 
that is being introduced in this course must be similar or at par with those in the industry. 


Guiding the students in learning the concept and theory of this course is very much relevant to the 
latest technology available in the market. The students develop a prototype Mobile Application similar 
to the industry standard. The course aims to guide the students to the industry standard environment in 
Mobile application development process. Hence, it is crucial to use authentic assessment in this course 
when it comes to the assessment. Dealing with real client exposes the students to the real industry 
expectation and client’s demand. The authentic assessment mirrors the actual process of developing a 
real mobile application. 


Background 


Over the years, assessment has been dominating factors of student’s failure and passing of the course. 
Such as important that passing the course assessment is a goal to one rather than the real 
understanding of the subject matter. In agreeing to this statement, Olson support the fact that a normal 
assessment required students to master, largely through memorization, specific contents, defined by 
textbooks and teachers (Olson, 2003). The value of the learning objectives is sadly diminished by the 
student’s eagerness to pass rather than understanding the content. In the case of a Digital Media 
program, remembering facts and memorization will not help in the contribution towards student’s 
competency their field. The program is structured to guide a competent student to develop a Digital 
Media content that remain relevant to the industry. In saying that, assessing digital media technology 
courses, the assessment could be a challenge because it links back to the proper use of the actual 
technology in the real industry. One should question on how students can apply the skill they have 
learned in the University and apply it at the workplace? 


“Assessment lies at the heart of the educator” (Little and Wolf, 1996; Ridgway et al, 2004) 
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Assessment is structured based on organized learning and teaching. The assessment that the students 
have completed might not reflect the real client’s expectation. Therefore, the authentic assessment 
comes in right in a position to justify the knowledge requirement for the students in preparation for 
real industry situation. In such authentic assessments, students apply the knowledge and skills of the 
discipline to situations or tasks that replicate real-world challenges. As Mueller(2014) reaffirmed that 
measurement of skills is indeed well suited to authentic assessment as it demonstrates skills acquired 
or developed to perform certain task. 


“The measurement of skills is particularly well suited to authentic assessment because the 
meaningful demonstration of skill acquisition or development requires a performance of 
some kind” (Mueller, 2014) 


It is safe to say that authentic assessment is also a problem-based learning as it assessed real client’s 
unique demand and situation. Ideally, Savin-Baden (2007) listed out the characteristics in problem- 
based learning which supports the features in authentic assessment: 


e Complex real-world situations that have no one ‘right’ answer are the organizing 
focus for learning. 


e Students work in teams to confront the problem, to identify learning gaps, and to 
develop viable solutions. 


e —_ Students gain new information through self-directed learning. 
e —— Staff act as facilitators 


e Problems lead to the development of clinical problem-solving capabilities. (Savin- 
Badin, 2007) 


This correlates well with the stated objectives of authentic assessment: 


“Authentic assessment, therefore, provides a sense of purpose for validation of their work 
that matches industry expectation during their study, as authentic assessment values the 
thinking behind work, the process, as much as the finished product” (Pearson & Valencia, 
1987; Wiggins, 1989; Wolf, 1989). 


Wiggins(1989) agrees that the worthiness of authentic assessment is beyond the value of learning a 
subject when he described authentic assessment as: 


“Engaging and worthy problems or questions of importance, in which students must use 
knowledge to fashion performances effectively and creatively. The tasks are either replica 
of or analogous to the kinds of problems faced by adult citizens and consumers or 
professionals in the field." (Wiggins, 1993, p. 229). 


Stiggins(1987) concurs that authentic assessment encourage the application of skills and knowledge 
the students have mastered, and it is then called upon students to perform specific skills and 
competencies (Stiggins, 1987, p. 34). 


Digital technologies have come a long way and transformed the way people communicate, in many 
aspects, it has predominant ones private and public life. The data or information transmitted in digital 
technology is called Digital media. The Digital Media content comes in various forms; it can be in a 
form of text, graphics, animation, video or audio. It is a digitized content that can be transmitted over 
the Internet or computer networks. 


In recent year with fast technology advances, mobile devices have taken over the role of the desktop 
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computer as a platform to download and deliver digital content. Mobile devices have played a large 
role in the digital media. 


“The use of Mobile phone or Smartphone has increased tremendously over the years. More 
people are also using their cell phones to send them updates from the Internet. Therefore, 
more people are using their cell phones for more than their original intended purposes. It 
has further broadened the ever-growing field of digital technology. The numbers of people 
using mobile devices are doubling by the months” (BFG1 14, 2011) 


Recognizing technologies changes in various usages is important as the steady grow of the 
combination of digital media, and mobile devices mean more opportunities in social activities, 
learning and work life will be uncovered in the future. Hague & William(2009) support the facts that 
technology and media must be challenged and questioned rather than accepted passively. ( Hague & 
Williamson, 2009) 


“With so many different types of content available, the question plaguing information 
experts today isn't what kind of content to create, but how to deliver it. With the continued 
popularity of smartphones, tablets, and eReaders, the average consumer now has a variety 
of options to turn to besides their computer to get the latest piece of digital content. 
Therefore, to be successful in the digital content realm, content creators must determine the 
best way to present their digital content to consumers. Maybe it is an ebook. Maybe it is an 
app. The right choice depends on the needs of the audience”. (Mullan, 2011) 


Based on this observation, there is a need of updating the way coursework and assessment is assessed 
as it is the essential part of the program. One should question, with the rapid technology changes, how 
and what sort of method that comes close to matching the real technology in the industry. A challenge 
for educator and course coordinator of the Digital Media course is to prepare the students to enter the 
real industry with adequate technology experience. 


Ben Reeves, head of the Australian Association of Graduate Employers (AAGE), in his statement 
explained, “Getting into university depends upon grades but getting into a job post-graduation depends 
on a range of other factors,” He added that factors like leadership, communication skills, and problem 
solving and customer service are essentials because these are the qualities that employers look for 
when hiring fresh graduate. These qualities aren’t necessary related to the candidate’s grades. 
(Callaghan, 2011). 


The best way to emulate the actual technology is by doing an assessment that helps the student 
understand the technology in use and the content needs of the current market. Therefore, authentic 
assessment plays an important role in Mobile Application Development course. 


Intervention 


To further access student’s assessment, an action research method is used to manage and monitor 
student’s performance. 


“Action research is the term that describes the integration of action (implementing a plan) 
with research (developing an understanding of the effectiveness of this implementation)” 
(Department of Education and Training, NSW 2010). 


The used of action research is highly recommended as it narrow down the action plan into smaller and 
manageable task (Mertler, C.A. & Charles, C.M., 2008). 


e It deals with their own problems, not someone else’s 
e It can start now—or whenever they are ready—providing immediate results 


e Action research provides them with opportunities to understand better, and, 
therefore, improve, their educational practices 
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e As a process, action research promotes the building of stronger relationships 
among staff 


e Importantly, action research provides educators with alternative ways of 
viewing and approaching educational questions providing a new way of 
examining their own practices. 


The assessment involves students taking part of understanding and managing client’s expectation. 
Apart from that they also need to know the latest technology that is compatible with the latest mobile 
devices. Proper guidelines are an important part of the assessment; students were given clear 
instruction of client’s requirements and design expectation. Access to client’s information and contact 
were available for the student as part of the routine procedure to get as much information as possible 
to help the development of the Mobile application. 


The course contained basic information on how to develop a simple, workable mobile application, the 
students then try to relate the course materials with the assessment that they were given. The course 
materials contained comprehensive programming language that were adjustable and easy to 
manipulate, the recommended APIs were available in the course materials, the students can also 
choose many available APIs online for their prototype. Most of these recommended APIs were 
standard used in the industry. 


The Mobile application designs were solely up to the students but as guidance, a few recommended 
online resources were given to the students for their inspirations. Standard use of Graphical User 
Interface GUI for mobile application were important, emphasizing in this area helps the student to 
understand the logic and rationale of using proper GUI for their Mobile Application. Since this 
assessment was for a real client, client’s input on the desired design and theme colour was taken into 
consideration in designing the mobile application interface design. The user’s perspective was also a 
part of the decision in designing the overall GUI. The student needs to work closely with the client as 
it will determine the success of the end result of the mobile application. 


The mobile application was intended for a specific group, the audiences for the mobile application 
were specific to group and interest. The students need to take this matter into account when designing 
the features of the mobile application. Taking into consideration the target user might not be 
technology savvy, the design needs to be simple but useful. Any added feature will either simplify or 
complicate the process of using the application. 


The students and the lecturer would then view the final product and present it to the real client. The 
real client gets to choose the best Mobile Applications that suite their needs in term of usability and 
designs. The selected Mobile Application will then be published and used. A notification about the 
status of the student's Mobile Application was sent to the students. 


Research Method 


After the completion of the authentic assessment, students were given online questionnaires. The 
questions asked were mainly targeting on how they feel about the assessment. Sixteen out of thirty- 
three targeted students responded to this survey. The questions were as following: 


e I find that I am more focused on this assessment because this assessment is for a real client. 
Otherwise, I would have just done it as normal assessment like the other courses. 

e I find that by doing this assessment, I have developed better understanding of the subject 
matter. 

e [can visualize the practicality of this course through the assessment because it deals with a 
real situation in the industry. 

e Jam motivated to work harder in this assessment more than in assessment for other courses 
because it is based on real client requirements. 
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e [find that I put in extra effort like reading and researching in the course because I wanted to 
perform my best. 


The five main online questions were presented to the students who had done the assessment. The 
students were given five options on each questions ranging from one to five with one as strongly 
disagree, five as strongly agree and three as neutral ground. Thirty-three students involved in this 
experiment and only sixteen students responded to the questionnaires. 


Result 


In the first question, the emphasis was on individual focus for authentic assessment for real client 
versus normal assessment. 56% responded with positive note while 19% feels that it did not make 
them focus even if the assessment is for the real client. 24% remain neutral on this survey. 


I find that |am more focused in this assessment because this 
assessment is for a real client. Otherwise | would have just done it 
as normal assessment like the other courses. 
8 
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Figure 1: Survey question one 
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The second question was more towards understanding of the subject matter. The question was 
designed to see if the students gain better understanding of the subject when dealing with real 
expectation. Interestingly, the survey shows that 75% feels that they have gained better understanding 
of the subject matter by doing real industry expectation task. 13% of the responded feels that they have 
gained nothing from the experience, and another 13% feels neither. 


I find that by doing this assessment, | have developed better 


understanding of the subject matter. 
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Figure 2: Survey question 2 


The third question deals with student realization of the worthiness of this assessment. Most of the 
respondent did realize the positive impact of the assessment in preparation for the real task in the 
industry, 63% were able to relate the practicality of the assessment with the real industry expectation. 
Only 13% feels that it is not practical, and 25% remain neutral. 


lam able to visualize the practicality of this course through the 
assessment because it deals with a real situation in the industry. 
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The fourth question touches on the motivational factor in doing the authentic assessment. 
Interestingly, 50% of the respondent feels motivated in working harder for the real client. 31% feels 
demotivated when it comes to dealing with real expectation, 19% is on the neutral ground when it 
comes to working with a real client. 


| am motivated to work harder in this assessment more than in 
assessment for other courses because it is based on real client 
requirements. 
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Figure 4: Survey question 4 


The last question was meant for student efforts, 69% has put on more effort in this assessment 
and did extra reading and researching for the task. 13% did not put in extra effort to do their 
best in the assessment, while 19% remain neutral. 


I find that | put extra effort like reading and researching in the 
course because | wanted to perform my best. 
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Figure 5: Survey question 5 
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Discussion 


Overall, the study shows interesting outcomes generated based on students response. More than 50% 
agrees on the positive impact of the authentic assessment in their course. Most of the students seem to 
work extra when it comes to working with a real client and the survey result shows students has been 
doing extra reading and research to perform better in the assessment. As Wiggins(1989) clearly stated 
that, the students must use knowledge to fashion performances effectively and creatively. To support 
this fact, 69% of the students participated in the survey were making extra effort in reading and doing 
research beyond the course content provided in the program. The students did this to achieve the best 
possible result. Based on the first question, the motivation behind doing extra reading and research 
seems to link to working with a real client. And by doing the extra work of this course, the student 
gain better understanding on the subject matter. 


The motivation may not be as high when it comes to following the real client’s expectation. The 
restriction of the creativity or freedom of expression in their work due to client’s strict guidelines, but 
the students realized the practicality of their assessment when it comes to the real industry expectation. 
Several authors concur that these skills include the development of creativity, self-motivation, 
innovation, problem-solving and collaboration skills. (McNeill, Gosper & Xu, 2012; Voogt, Erstad, 
Dede & Mishra, 2013; Kaufman, 2013) These are also skills that were developed by students in a 
problem-based learning context. 


Validation of their work were important as 63% of the students surveyed students believes what they 
were doing mirror the expectation in the industry. Pearson & Valencia (1987) validated this statement 
as authentic assessment values the thinking behind work, the process, as much as the finished product. 
The idea of working for real client enables students to visualize the practicality of the authentic 
assessment in the real word. 


In line with Mueller(2014), this is considered good as the authentic assessment because students were 
exposed to the real technologies that are in used in the real industry. Students face a real problem in 
dealing with the real client that almost certainly complicates the process of development of the 
application. By experiencing the phases in the development of Mobile Application for the real market, 
the students will benefit in gaining knowledge of dealing with advance and real-time technology and 
expectation. 


As Savin-Baden (2007) listed out in the characteristic in problem-based learning, the list indeed 
applicable when it comes to authentic assessment. It allows the students to be exposed to the complex 
real situation; it gives them challenge to be more creative and dynamic in solving a problem. In some 
cases, students are required to work in a team to discuss and solve the large collective problem. In 
dealing with the real life situation, the students are forced to gain new information by investigation 
and doing research to solve individual cases in the assessment. The benefits of self-directing learning 
are enhanced confident in the subject matter and the students were able to visualize the concept in a 
bigger picture. The role of a lecturer, in this case, is mainly a facilitator; this gives the student 
empowerment of what they can do with the knowledge they have gained through self-learning. In the 
end, it leads to better understanding of the situation that the students were in and the development of 
problem-solving capabilities. 


Conclusion 


The benefit gained in the implementation of authentic assessment in the Mobile Application 
Development course is favorable. It was proven that the students who participated in this survey 
acknowledge the benefits of doing an assessment for real client. They were more focused on doing 
their best in the assessment. The students were able to visualize the real industry expectation while 
doing the assessment. Extra reading and research were done on their own effort, and in return they 
gained more understanding on the subject matter. It has given them good motivation because they 
know what they were doing and the direction they were heading. 
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Overall, the inclusion of authentic assessment in one of the Bachelor of Digital Media unit course has 
proven to be a positive move. It has given the important insight of the real expectation of the industry. 
A technology-based program like Bachelor of Digital Media requires more of this type of assessment. 
The pilot study was the first move into the overall implementation of the authentic assessment in the 
program. The real benefits of the authentic assessment go to the students as it not only prepares them 
to the real industry in the future, but it also helps them to acquire adequate skill and knowledge for the 
real world out there. And to say the least, it is what the employers are looking for in their future 
employee. 
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